A multi-sensor benchmark dataset for detecting individual trees in airborne RGB, Hyperspectral and LIDAR point clouds
Individual tree detection is a central task in forestry and ecology. Few papers analyze proposed methods across a wide geographic area. The NeonTreeEvaluation dataset is a set of bounding boxes drawn on RGB imagery for 22 sites in the National Ecological Observation Network (NEON). Each site covers a different forest type(e.g. TEAK). This dataset is the first to have consistant annotations across a variety of ecosystems for co-registered RGB, LiDAR and hyperspectral imagery.
Evaluation images are included in this repo under /evaluation folder. Annotation files (.xml) are included in this repo under /annotations/
For the larger annotated training tiles, as well as unannotated training tiles for additional sites, see zenodo archive
Mantainer: Ben Weinstein - University of Florida.
How do I evaluate against the benchmark?
We have built an R package for easy evaluation and interacting with the benchmark evaluation data.
How were images annotated?
Each visible tree was annotated to create a bounding box that encompassed all portions of the vertical object. Fallen trees were not annotated. Visible standing stags were annotated.
For the point cloud annotations, the two dimensional bounding boxes were draped over the point cloud, and all non-ground points (height < 2m) were excluded. Minor cosmetic cleanup was performed to include missing points. In general, the point cloud annotations should be seen as less thoroughly cleaned, given the tens of thousands of potential points in each image.
Sites (NEON locations)
| siteID, State | Forest Description | Evaluation Annotations | Training Annotations |
|---|---|---|---|
| SJER, CA | Oak Savannah | 293 | 2533 |
| TEAK, CA | |||
| NIWO, CO |
How can I add to this dataset?
Anyone is welcome to add to this dataset by cloning this repo and labeling a new site in rectlabel. NEON data is available on the NEON data server. We used the NEON 2018 “classified LiDAR point cloud” data 104 product (NEON ID: DP1.30003.001), and the “orthorectified camera mosaic” (NEON ID: 105 DP1.30010.001). Please follow the current folder structure, with .laz and .tif files saved together in a single folder, with a unique name, as well as a single annotations folder for the rect label xml files. See /SJER for an example.
For ease of access, we have added two unlabeled sites, BART, and UNDE, we encourage others to label these sites, or use models from the labeled data to predict into new, untested, areas.
RGB
library(raster)
library(NeonTreeEvaluation)
#Read RGB image as projected raster
rgb_path<-get_data(plot_name = "SJER_021",sensor="rgb")
rgb<-stack(rgb_path)
#Path to dataset
xmls<-readTreeXML(siteID="SJER")
#View one plot's annotations as polygons, project into UTM
#copy project utm zone (epsg), xml has no native projection metadata
xml_polygons <- boxes_to_spatial_polygons(xmls[xmls$filename %in% "SJER_021.tif",],rgb)
plotRGB(rgb)
plot(xml_polygons,add=T)
Lidar
To access the draped lidar hand annotations, use the “label” column. Each tree has a unique integer.
library(lidR)
path<-get_data("TEAK_052",sensor="lidar")
r<-readLAS(path)
trees<-lasfilter(r,!label==0)
plot(trees,color="label")
The same is true for the training tiles (see below)
We elected to keep all points, regardless of whether they correspond to tree annotation. Non-tree points have value 0. We recommend removing these points before evaluating the point cloud. Since the annotations were made in the RGB and then draped on to the point cloud, there will be some erroneous points at the borders of trees.
Hyperspectral
path<-get_data("MLBS_071",sensor="hyperspectral")
g<-stack(path)
nlayers(g)
[1] 426
#Grab a three band combination to view as false color
g<-g[[c(17,55,113)]]
nlayers(g)
[1] 3
plotRGB(g,stretch="lin")
And in the training data:
Training Tiles
We have uploaded the large training tiles to Zenodo for download.
This includes
The annotated trainings tiles (optionally cropped) for the NIWO, MLBS, SJER, TEAK, LENO, and OSBS sites. These site training tiles vary in size.
Unannotated training tiles for the 15 additional sites. Training tiles do not overlap with evaluation plots.
TODO add zenodo link.
Performance
To submit to this benchmark, please see
https://github.com/weecology/NeonTreeEvaluation_package
The primary evaluation statistic is precision and recall across all sites. It is up to the authors to choose the best probability threshold if appropriate.
| Author | Precision | Recall | Description | |
|---|---|---|---|---|
| Weinstein et al. 2019 1 | 0.55 | 0.65 | Semi-supervised RGB Deep Learning | |
| Silva et al. 2016 | 0.09 | 0.23 | Unsupervised LiDAR raster | |
| Dalponte et al 2016 | Unsupervised LidAR raster | |||
| Li et al. 2012 | Unsupervised LiDAR point cloud |
To reproduce the analysis for the benchmark comparison see https://github.com/weecology/NeonTreeEvaluation_analysis
Cited
1 Weinstein, Ben G., et al. “Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks.” Remote Sensing 11.11 (2019): 1309. https://www.mdpi.com/2072-4292/11/11/1309 Thanks to the lidR R package for making algorithms accessible for comparison.
Please submit a pull request, or contact the mantainer if you use these data in analysis and would like the results to be shown here.